Abstract Now point cloud semantic segmentation is widely applied in various fields such as autonomous driving and virtual reality. However, the current point cloud semantic segmentation algorithms cannot extract relatively complete spatial structure information, and the information for each point is difficult to explain. To address this deficiency, a 3D point cloud semantic segmentation network based on coding feature learning is proposed. Firstly, the local feature encoder is designed based on the introduction of angle information and the enhanced features to learn more complete local spatial structures and alleviate the problem of misclassification of similar objects. Secondly, mixed pooling polymerization module is designed to aggregate rough features and fine features while ensuring the sorting invariance of point cloud. Finally, the multi-scale feature fusion is adopted to fully utilize the different scale features in the encoding layer and achieve accurate semantic segmentation. The experiment on two large benchmark datasets, S3DIS and SemanticKITTI, demonstrates the superiority of the proposed network.
Fund:National Key R&D Plan Project(No.2019YFB1309905,2020YFB1712802)
Corresponding Authors:
TONG Guofeng, Ph.D., professor. His research interests include computer vision, 3D urban reconstruction and deep learning.
About author:: LIU Yongxu, master student. His research interests include computer vision, laser point cloud processing and deep learning.PENG Hao, Ph.D. candidate. His research interests include computer vision, laser point cloud processing and pattern recognition.SHAO Yuyuan, Ph.D. candidate. His research interests include computer vision, laser point cloud processing and pattern recognition.
TONG Guofeng,LIU Yongxu,PENG Hao等. 3D Point Cloud Semantic Segmentation Network Based on Coding Feature Learning[J]. Pattern Recognition and Artificial Intelligence, 2023, 36(4): 313-326.
[1] GUO Y L, WANG H Y, HU Q Y, et al. Deep Learning for 3D Point Clouds: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(12): 4338-4364. [2] KOPPULA H S, ANAND A, JOACHIMS T, et al. Semantic Labeling of 3D Point Clouds for Indoor Scenes[C/OL].[2023-01-05]. https://www.cs.cornell.edu/~hema/papers/NIPS2011_0186.pdf. [3] LI R K, ZHANG Y M, NIU D M, et al. PointVGG: Graph Convolutional Network with Progressive Aggregating Features on Point Clouds. Neurocomputing, 2020, 429: 187-198. [4] SHELHAMER E, LONG J, DARRELL T.Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. [5] QI C R, SU H, MO K C, et al. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 77-85. [6] QI C R, YI L, SU H, et al. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 5105-5114. [7] HU Q Y, YANG B, XIE L H, et al. RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 11108-11114. [8] THOMAS H, QI C R, DESCHAUD J E, et al. KPConv: Flexible and Deformable Convolution for Point Clouds // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 6410-6419. [9] XU M T, DING R Y, ZHAO H S, et al. PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 3172-3181. [10] WANG S L, SUO S, MA W C, et al. Deep Parametric Continuous Convolutional Neural Network // Proc of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 2589-2597. [11] LANDRIEU L, BOUSSAHA M.Point Cloud Oversegmentation with Graph-Structured Deep Metric Learning // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 7440-7449. [12] KANG Z H, LI N.PyramNet: Point Cloud Pyramid Attention Network and Graph Embedding Module for Classification and Segmentation. Australian Journal of Intelligent Information Processing Systems, 2019, 16(2): 35-43. [13] YIN Z, TUZEL O.VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 4490-4499. [14] YIN J B, SHEN J B, GUAN C Y, et al. LiDAR-Based Online 3D Video Object Detection with Graph-Based Message Passing and Spatiotemporal Transformer Attention // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 11492-11501. [15] LONG A H, VORA S, CAESAR H, et al. PointPillars: Fast Encoders for Object Detection from Point Clouds // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 12689-12697. [16] GEIGER A, ULUSOY A O, GERNOT R.OctNet: Learning Deep 3D Representations at High Resolutions // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 6620-6629. [17] SHI S S, WANG X G, LI H S.PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 770-779. [18] MENG Q H, WANG W G, ZHOU T F, et al. Towards a Weakly Supervised Framework for 3D Point Cloud Object Detection and Annotation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(8): 4454-4468. [19] ZHOU D F, FANG J, SONG X B, et al. Joint 3D Instance Segmentation and Object Detection for Autonomous Driving // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 1836-1846. [20] SU H, MAJI S, KALOGERAKIS E, et al. Multi-view Convolutional Neural Networks for 3D Shape Recognition // Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2015: 945-953. [21] BOULCH A, LE SAUX B, AUDEBERT N, et al. Unstructured Point Cloud Semantic Labeling Using Deep Segmentation Networks // Proc of the Workshop on 3D Object Retrieval. New York, USA: ACM, 2017: 17-24. [22] GRAHAM B, ENGELCKE M, VAN DER MAATEN L, et al. 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 9224-9232. [23] ZHAO H S, JIANG L, JIA J Y, et al. Point Transformer // Proc of the IEEE International Conference on Computer Vision. Wa-shington, USA: IEEE, 2021: 16259-16268. [24] 詹艳艳,徐荣聪,陈晓云.基于插值边缘算子的时间序列模式表示.模式识别与人工智能, 2007, 20(3): 421-427. (ZHAN Y Y, XU R C, CHEN X Y.Time Series Pattern Representation Based on Interpolated Edge Operator. Pattern Recognition and Artificial Intelligence, 2007, 20(3): 421-427.) [25] ARMENI I, SENER O, ZAMIR A R, et al. 3D Semantic Parsing of Large-Scale Indoor Spaces // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 1534-1543. [26] BEHLEY J, GARBADE M, MILIOTO A, et al. SemanticKITTI: A Dataset for Semantic Scene Understanding of Lidar Sequences // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 9296-9306. [27] HUANG Q G, WANG W Y, NEUMANN U.Recurrent Slice Networks for 3D Segmentation of Point Clouds // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 2626-2635. [28] LANDRIEU L, SIMONOVSKY M.Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 4558-4567. [29] LI Y Y, BU R, SUN M C, et al. PointCNN: Convolution on χ-Transformed Points // Proc of the 32nd International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2018: 828-838. [30] ZHAO H S, JIANG L, FU C W.PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019: 5560-5568. [31] ZHANG Z Y, HUA B S, YEUNG S K.ShellNet: Efficient Point Cloud Convolutional Neural Networks Using Concentric Shells Statistics // Proc of the IEEE/CVF International Conference on Computer Vision. Washington, USA: IEEE, 2019: 1607-1616. [32] YAN X, ZHENG C D, LI Z, et al. PointASNL: Robust Point Clouds Processing Using Nonlocal Neural Networks with Adaptive Sampling // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 5589-5598. [33] FAN S Q, DONG Q L, ZHU F H, et al. SCF-Net: Learning Spatial Contextual Features for Large-Scale Point Cloud Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pa-ttern Recognition. Washington, USA: IEEE, 2021: 14499-14508. [34] TANG L Y, ZHAN Y B, CHEN Z, et al. Contrastive Boundary Learning for Point Cloud Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 8479-8489. [35] MILIOTO A, VIZZO I, BEHLEY J, STACHNISS C, et al. RangeNet++: Fast and Accurate LiDAR Semantic Segmentation // Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington, USA: IEEE, 2019: 4213-4220. [36] ZHANG Y, ZHOU Z X, DAVID P, et al. PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 9598-9607. [37] FANG Y, XU C Y, CUI Z, et al. Spatial Transformer Point Convolution[C/OL].[2023-01-05]. https://arxiv.org/pdf/2009.01427.pdf. [38] ALNAGGAR Y A, AFIFI M, AMER K, et al. Multi Projection Fusion for Real-Time Semantic Segmentation of 3D LiDAR Point Clouds // Proc of the IEEE Winter Conference on Applications of Computer Vision. Washington, USA: IEEE, 2021: 1799-1808.
2023, Vol. 36 
